Repository navigation
fix(ci): drop byte-identity gates that fail on dependency bumps - #176
Conversation
[Medium impact] The PR appears safe to merge within its stated scope of removing committed-byte checks. SummaryRemoves committed-byte comparisons from native builds, binding generation, and schema generation.
auto-wood explicitly describes the removed byte, size, and recorded-hash checks as intentional. No separate actionable regression was found. Reviews (1) · Last reviewed commit: "fix(ci): drop byte-identity gates that f..." · Reviewed by Greptile |
✅ Verdict:
|
| Findings | Count |
|---|---|
| 🔴 must-fix | 0 |
| 🟠 should-fix | 0 |
| 🟡 may-fix | 1 |
| 🔵 nit | 0 |
📋 Review
What this PR does: This PR removes validation that asserts previously committed artifact bytes instead of behavior: the xcframework committed-vs-rebuilt archive comparison (the whole two-tree mode), uniffi Swift/Python regeneration byte diffs, codegen git diff on generated trees plus the generated-Rust size bound, and C++ committed provenance hash verification in both carrier lanes. The stated root cause is that dependency-only Cargo.lock bumps (e.g. uuid 1.29.0→1.29.2, sha2 0.11.0→0.11.1) rebuild behaviorally identical archives and fail the required xcframework gate with drift detected. Behavior coverage is kept: C ABI symbol checks, C++/Kotlin/C#/Swift smokes, golden vectors, schema count, opaque-JSON type contract, lockstep version surfaces, and the apply script's transfer-manifest verification before staging.
Findings
🟡 Xcframework refresh text now claims builds are never bit-reproducible, contradicting the repo's same-input reproducibility contract
-
Evidence:
[DOCS-01] Blanket non-reproducibility claim contradicts the documented same-input check
- Evidence:
CONTRIBUTING.md:124— the edited closing sentence now reads "Builds are not bit-reproducible across runs; the transfer manifest and producer record identify the applied bytes."; it replaced "A fresh build may differ from a later rebuild; retain the actual two-tree comparison and investigate drift rather than treating successful application as reproducibility" (pr.diff, hunk@@ -121,13 +121,13 @@). - Evidence:
.agents/knowledge/tooling-decisions/ci-assembled-committed-native-artifacts.md:52— "Independent reproducibility requires another real source build with equivalent inputs." - Evidence:
.agents/knowledge/tooling-decisions/ci-assembled-committed-native-artifacts.md:58— "For a focused reproducibility check, retain the first output, clean the task-owned build directory, and compile again at the exact same path with the same inputs. Compare complete raw outputs …";:66— "Independent cold builds and whole-file mismatch controls establish reproducibility within their declared input boundary." - Impact: The maintainer guide now states as fact that bit-reproducibility is unattainable, while the repo's tooling-decision record prescribes exactly the two-build, same-path comparison as the way to establish it and still tells a native comparison to distinguish changed inputs from same-input nondeterminism (
…native-artifacts.md:78). The removed clause was the only CONTRIBUTING statement keeping that comparison/drift-investigation in the refresh path, so a maintainer following this file is steered away from the check the knowledge record requires (and from treating a byte-identical control as evidence). - Attack result (three-way): counter-example partly holds — separate CI runs use different absolute output paths, so
LC_ID_DYLIB/UUID/section offsets normally differ andCHANGELOG.md:541already gates Windows freshness on build revisi…
- Evidence:
-
Location:
CONTRIBUTING.md:124 -
Category: docs
-
Merge class: may-fix
Revalidated findings
Reassessed 8/9 prior finding(s) this round · 0 invalid-output · 1 omitted · 0 over cap · 9 recheckable · context coverage: truncated
| finding | disposition | evidence | thread |
|---|---|---|---|
CONTRIBUTING still advertises git diff on generated trees in verify-codegen |
addressed (verified-fix) | current-code | - |
| Seat coverage declaration (not a finding) | dismissed (non-fix-dismissal) | current-code | - |
| Committed xcframework bundle now has no verifier against its committed manifest/provenance sidecar | unverifiable (no-evidence) | - | - |
| verify-codegen no longer asserts the committed generated trees match a fresh regeneration | dismissed (non-fix-dismissal) | current-code | - |
| Codegen knowledge still asserts a zero-drift proof covers generated bytes | addressed (verified-fix) | current-code | - |
cpp-build.mjs --verify removed, leaving committed C carrier hashes unenforced while docs still demand they match |
dismissed (non-fix-dismissal) | current-code | - |
| Committed C/C++ carrier provenance hashes are no longer verified anywhere | dismissed (non-fix-dismissal) | current-code | - |
| xcframework refresh instructions still require a two-tree comparison whose tool was deleted | addressed (verified-fix) | current-code | - |
| verify-codegen no longer detects committed generated-tree drift | dismissed (non-fix-dismissal) | current-code | - |
Linked-issue AC
none — the PR links no issue (issues is empty; #171/#172/#173/#175 appear only as open dependency PRs expected to be rebased, not as linked acceptance criteria).
✅ Verified
- CONTRIBUTING.md:124 was read directly: the refresh paragraph now closes with "Builds are not bit-reproducible across runs; the transfer manifest and producer record identify the applied bytes."
- The replaced clause was confirmed from pr.diff hunk @@ -121,13 +121,13 @@, which shows the removed "retain the actual two-tree comparison and investigate drift" sentence.
- .agents/knowledge/tooling-decisions/ci-assembled-committed-native-artifacts.md:52,58,66 still require an independent source build with equivalent inputs and a complete raw-output comparison at the same path, and :78 still lists "distinguish changed inputs from same-input nondeterminism" as a when-to-apply condition.
- verify-codegen row checked against package.json:15: the codegen script chains generated code with assert-schema-count.mjs, assert-opaque-json-types.mjs, the deep-ref-hoist node test and cargo check --workspace; no generated-tree git diff gate exists under the worktree, so the row correction is accurate.
- C-carrier refresh sentence checked against .github/workflows/cpp-connect.yml:156-168 and tooling/connect/cpp-build.mjs:298-312: the Windows lane uploads the provenance.json the build rewrote, so the replacement text is accurate.
- Prior-finding revalidation: three findings resolved as verified-fix (CONTRIBUTING git diff advertisement, codegen zero-drift knowledge assertion, xcframework two-tree refresh instructions); four candidates dismissed as non-fix.
- ❓ PR-body verification claims (22 tests across cpp-build.test.mjs and apply-xcframework-artifact.test.mjs, pnpm run codegen + schema/opaque/deep-ref-hoist, cargo +nightly check --workspace, twin-parity.mjs) are not evidenced by the review document.
🗑️ Considered & rejected
- verify-codegen no longer asserts a fresh regeneration matches the committed generated trees — dismissed as non-fix: the removed snapshot comparison is the exact gate class this PR intends to drop, and it fails on dependency-only bumps without behavior change.
- Codegen knowledge still asserted that a zero-drift proof covers generated bytes — revalidated as addressed (verified-fix), so no finding was raised.
- cpp-build.mjs --verify removed while maintainer docs still demand committed C carrier hashes match — dismissed as non-fix: the documented refresh path sources provenance.json from the lane build that rewrote it (cpp-connect.yml:156-168), so docs and scripts agree.
- Committed C/C++ carrier provenance hashes are no longer verified anywhere — dismissed as non-fix, same reasoning: transfer identity comes from the refreshed producer record plus the committed carrier ABI/smoke checks, not a stale hash.
- xcframework refresh instructions requiring a two-tree comparison whose tooling was deleted — revalidated as addressed at this head.
- Seat coverage declaration — dismissed as a non-finding (process note, not a defect in the diff).
- Omission of the deep-ref-hoist unit-test step from the verify-codegen documentation row — dropped as non-admissible polish.
- CHANGELOG.md:541 gating Windows freshness on build revision rather than byte equality was considered as support for the new wording and partly accepted: separate runs use different absolute output paths, so the blanket claim is defensible for cross-run artifacts; only the unqualified quantifier remains a defect.
🛠️ Plan to fix
Expand fix plan
Only DOCS-01 (`may-fix`, XS, LOW risk) remains. No code, CI or artifact behavior depends on the wording.
1. **Edit `CONTRIBUTING.md:124`** (the closing sentence of "Refreshing the Swift xcframework"): replace the blanket claim with a qualified one, e.g. "Artifacts from separate CI runs are not byte-identical — different absolute build paths change `LC_ID_DYLIB`, UUIDs and section offsets — so the transfer manifest and producer record identify the applied bytes. To establish reproducibility, rebuild at the same path with identical inputs and compare complete raw outputs." Keep the manifest + producer-record identity sentence; keep the pointer to the same-input two-tree comparison.
2. **Confirm parity with the knowledge record** in `.agents/knowledge/tooling-decisions/ci-assembled-committed-native-artifacts.md` § "Compare independent builds with identical complete inputs" (`:52-66`) and the `:78` when-to-apply entry; the new wording must not read as retiring the same-input obligation. Do not edit that file — it is already correct.
3. **Check the adjacent `:46` statement** ("apply a validated refresh without comparing two build trees") still reads as application identity, not as a reproducibility claim, so the two sentences do not conflict.
4. **Verification gates**: re-read the refresh section for self-consistency with `CHANGELOG.md:541` (build-revision freshness, not byte equality); run `node tooling/docs/twin-parity.mjs` only if the touched section is mirrored in the docs site. No build/test matrix is required — the change is prose inside a maintainer doc.Dismissed because a newer commit was pushed; Greptile will re-review the current head.
|
All six review threads resolved.
Verification of the original failure class: #171 ( |
Root cause
The required
xcframeworkcheck compared the committed archives (crates/spoke-connect/bindings/swift/xcframework/spoke_connectFFI.xcframework/*.a) byte-for-byte against a fresh build. AnyCargo.lockbump that touchesspoke-connect's dependency tree changes the compiled archives, so dependency-only PRs fail withdrift detectedwithout any behavior change — #171 (uuid1.29.0 → 1.29.2) and #172 (sha20.11.0 → 0.11.1).The same class of snapshot coverage existed in three other places: bindgen regeneration byte diffs, generated-tree
git diffplus a generated-Rust size bound, and C++ committed-provenance hash verification.Change
Remove coverage that asserts previously committed bytes, sizes or record hashes instead of behavior:
git diff --exit-codeon generated trees and the generated Rust size boundKept as behavior coverage: C ABI symbol checks, C++/Kotlin/C#/Swift smokes, golden vectors, schema count, opaque-JSON type contract, lockstep version surfaces. The xcframework transfer manifest remains;
apply-xcframework-artifact.shstill checksum-verifies the downloaded bundle before staging.Effect on open dependency PRs
After merge, rebasing #171/#172/#173/#175 onto
maindrops these gates from their runs. A remaining failure on those branches would be a real behavior or compatibility failure (e.g. the TypeScript 7 major bump), not artifact byte drift.Verification
node --test tooling/connect/cpp-build.test.mjs tooling/connect/apply-xcframework-artifact.test.mjs— 22 passpnpm run codegen+ schema count + opaque-JSON + deep-ref-hoist test — pass; generated trees unchangedcargo +nightly check --workspace— passnode tooling/docs/twin-parity.mjs— OK